Outline

1. Multilayer Perceptron (MLP) Options
2. Error Functions


1. Multilayer Perceptron (MLP) Options

a. Train an MLP using backpropagation (BP). Batching, which 
   denotes the accumulation of weight changes over portions of the 
   training set before they are used, is an option. The learning 
   factor changes adaptively. 

b. Fast training of MLP networks. Trains networks one or two 
   orders of magnitude faster than BP.

c. Analyze and prune trained MLPs from BP or fast training.
   Produces weight and network structure files for the pruned network,
   which can be saved to disk, in the non-demo version.

d. Process data using a trained MLP. Data may or may not include 
   desired outputs.

e. Create MLP subroutine. Given a network structure file and a 
   weight file, creates an MLP subroutine in Fortran with a parameter 
   list that includes only the input array and output array.

f. Create formatted weight file. Given a network structure file and a 
   weight file, creates a formatted weight file that clearly shows 
   the different connections and their weights and thresholds.


2. Error Functions

 a. The error function that is being minimized during backpropagation
    training and fast training is

                   Nout      
    MSE = (1/Npat) SUM MSE(k)     where
                   k=1  

              Npat              2
    MSE(k) =  SUM [ Tpk - Opk ]
              p=1  

    where Npat is the number of training patterns, Nout is the number 
    of network output nodes, Tpk is the desired output for the pth
    training pattern and the kth output, and Opk is the actual output
    for the pth training pattern and the kth output. MSE is printed
    for each iteration.

 b. Additional errors printed out are defined as follows.
    The rms error of the kth output, RMS(k), is SQRT( MSE(k)/Npat ),
    where SQRT means square root.

    The kth output's Relative RMS Error is

    R(k) = SQRT( MSE(k)/E(k) ) where

            Npat           2
    E(k) =  SUM [ Opk-Mk ]      and
            p=1  

                  Npat 
    Mk = (1/Npat) SUM  Opk 
                  p=1  

    The kth output's Error Variance is MSE(k)/Npat.
